Skip to content

feat: adaptive zoom intensity based on click spatial density#1666

Open
namearth5005 wants to merge 9 commits intoCapSoftware:mainfrom
namearth5005:feat/auto-zoom-intensity
Open

feat: adaptive zoom intensity based on click spatial density#1666
namearth5005 wants to merge 9 commits intoCapSoftware:mainfrom
namearth5005:feat/auto-zoom-intensity

Conversation

@namearth5005
Copy link

@namearth5005 namearth5005 commented Mar 18, 2026

Summary

Makes auto-zoom smarter by adapting zoom level to click cluster density (refs #1646, builds on #1663, #1664, #1665):

  • Tight click clusters zoom more — clicks close together (e.g., UI button rapid interactions) get up to 2.5x zoom for clear focus
  • Spread activity zooms less — clicks spanning the screen get 1.2x zoom so context is preserved
  • Linear interpolation between min/max based on max pairwise distance relative to intensity_spatial_scale
  • Backward compatible — when min_zoom_amount == max_zoom_amount, behavior is identical to the flat zoom_amount

Algorithm

spread = max pairwise distance between clicks in cluster
t = clamp(spread / intensity_spatial_scale, 0, 1)
zoom = max_zoom + t * (min_zoom - max_zoom)

Config fields added to AutoZoomConfig

Field Type Default Purpose
min_zoom_amount f64 1.2 Minimum zoom for spread-out activity
max_zoom_amount f64 2.5 Maximum zoom for tight clusters
intensity_spatial_scale f64 0.3 Spread distance at which zoom hits minimum

Implementation

Changed interval tracking from Vec<(f64, f64)> to Vec<(f64, f64, Vec<(f64, f64)>)> to carry click positions through the merge step. The compute_zoom_amount function computes per-segment zoom from position data.

UI changes

Replaced single "Zoom Amount" slider with "Min Zoom" (1.0-3.0) and "Max Zoom" (1.5-4.0) sliders in experimental settings.

Test plan

  • intensity_tight_cluster_zooms_more — 3 clicks within 0.02 spread → zoom > 2.0
  • intensity_spread_activity_zooms_less — 3 clicks spanning 0.8 → zoom < 1.8
  • intensity_disabled_when_equal — min == max == 1.5 → all segments get exactly 1.5x
  • Manual: click rapidly on small button → deep zoom; navigate across screen → subtle zoom

Greptile Summary

This PR introduces adaptive zoom intensity for auto-zoom-on-clicks: click clusters with tight spatial spread now receive up to 2.5× zoom while spread-out activity scales down to 1.2×, computed via max pairwise distance and linear interpolation against a configurable intensity_spatial_scale. It also expands AutoZoomConfig with dead-zone merging, double-click deduplication, and right-click filtering, and exposes Min/Max Zoom sliders in experimental settings.

Key changes:

  • AutoZoomConfig struct added in configuration.rs with 17 configurable fields and proper serde backward-compatibility annotations on the three new fields.
  • intervals type widened from Vec<(f64, f64)> to Vec<(f64, f64, Vec<(f64, f64)>)> to carry click positions through the merge step, enabling per-segment zoom computation.
  • New inner compute_zoom_amount function in recording.rs implements the density-based interpolation formula.
  • generate_zoom_segments_from_clicks Tauri command now loads GeneralSettingsStore to pass the config along.
  • Missing min ≤ max validation: the UI allows dragging "Min Zoom" above "Max Zoom" (e.g. 3.0 vs 1.5), which silently inverts the density mapping — tight clusters get lower zoom than spread activity.
  • NaN zoom risk: compute_zoom_amount divides by intensity_spatial_scale without guarding against zero; if the scale is manually set to 0.0 and all cluster positions are identical, the result is NaN and propagates to ZoomSegment.amount.

Confidence Score: 3/5

  • Needs fixes for inverted-zoom UI bug and NaN guard before merging.
  • The algorithm and data-structure changes are sound and well-tested, but two actionable issues lower confidence: (1) the absence of a min ≤ max constraint in the UI means users can easily invert the adaptive behavior by accident, and (2) a missing zero-guard in compute_zoom_amount can produce a NaN zoom amount that would silently break the rendered segment for manually edited configs.
  • apps/desktop/src-tauri/src/recording.rs (NaN guard in compute_zoom_amount) and apps/desktop/src/routes/(window-chrome)/settings/experimental.tsx (Min/Max Zoom slider ordering validation).

Important Files Changed

Filename Overview
apps/desktop/src-tauri/src/recording.rs Core zoom-segment generation refactored to accept AutoZoomConfig; new compute_zoom_amount inner function implements adaptive zoom via max pairwise distance; intervals now carry click positions through the merge step. Two issues found: NaN zoom when intensity_spatial_scale=0 & max_dist=0, and the dead-zone OR condition can silently merge temporally distant clicks.
crates/project/src/configuration.rs New AutoZoomConfig struct added with serde camelCase and struct-level #[serde(default)]; the three new fields (min_zoom_amount, max_zoom_amount, intensity_spatial_scale) correctly add explicit per-field #[serde(default = "fn")] for backward-compatibility with existing stored configs. The inconsistency (other fields lack these annotations) is a future-maintenance concern but not an immediate bug.
apps/desktop/src/routes/(window-chrome)/settings/experimental.tsx New SettingSlider component and handleConfigChange helper added; Min Zoom and Max Zoom sliders exposed for the new adaptive zoom feature. Missing constraint enforcement allows min_zoom > max_zoom, which inverts the density-based zoom behavior.
apps/desktop/src-tauri/src/lib.rs generate_zoom_segments_from_clicks Tauri command updated to load GeneralSettingsStore and pass auto_zoom_config to the recording module; error handling uses double-unwrap-or-default pattern which silently falls back to defaults on store read failure.
apps/desktop/src-tauri/src/general_settings.rs auto_zoom_config: AutoZoomConfig field added to GeneralSettingsStore with #[serde(default)]; Default impl updated to match. Straightforward and correct.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[generate_zoom_segments_from_clicks\nTauri command] --> B[Load GeneralSettingsStore\nunwrap_or_default]
    B --> C[generate_zoom_segments_for_project\nmeta + recordings + AutoZoomConfig]
    C --> D[generate_zoom_segments_from_clicks_impl]

    D --> E{ignore_right_clicks?}
    E -- yes --> F[Filter cursor_num != 0]
    E -- no --> G[Sort clicks by time_ms]
    F --> G

    G --> H{double_click_threshold_ms > 0?}
    H -- yes --> I[Deduplicate rapid same-button clicks]
    H -- no --> J[Build click_groups]
    I --> J

    J --> K{For each unprocessed click:\ntime_and_spatial OR in_dead_zone?}
    K -- yes --> L[Append to existing group]
    K -- no --> M[Create new group]
    L & M --> N[Convert groups → intervals\ncarrying click positions]

    N --> O[Add movement intervals\npositions = empty vec]
    O --> P[Sort & Merge intervals\nextend positions on merge]

    P --> Q[compute_zoom_amount\nper merged segment]
    Q --> R{max == min?}
    R -- yes --> S[return zoom_amount\nbackward-compat]
    R -- no --> T{positions.len < 2?}
    T -- yes --> U[return max_zoom_amount]
    T -- no --> V[Compute max pairwise distance]
    V --> W[t = clamp\nmax_dist / intensity_spatial_scale]
    W --> X[zoom = max + t × min - max]
    X --> Y[ZoomSegment\nstart/end/amount]
    S & U --> Y
Loading

Comments Outside Diff (2)

  1. apps/desktop/src/routes/(window-chrome)/settings/experimental.tsx, line 793-810 (link)

    P1 No constraint enforcing Max Zoom ≥ Min Zoom

    The "Min Zoom" slider has a range of 1.0–3.0 and "Max Zoom" starts at 1.5–4.0, so a user can drag Min Zoom above Max Zoom (e.g. minZoomAmount=3.0, maxZoomAmount=1.5). In compute_zoom_amount, the formula is:

    config.max_zoom_amount + t * (config.min_zoom_amount - config.max_zoom_amount)
    

    When min > max, (min - max) is positive, so t increasing with spread increases zoom instead of decreasing it. This silently inverts the intended behavior: tight clusters would receive low zoom (1.5×) while spread activity receives high zoom (3.0×) — the opposite of the feature description. There is no guard in the backend either.

    Consider adding a UI-level validation that clamps or prevents the inverted state:

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: apps/desktop/src/routes/(window-chrome)/settings/experimental.tsx
    Line: 793-810
    
    Comment:
    **No constraint enforcing Max Zoom ≥ Min Zoom**
    
    The "Min Zoom" slider has a range of 1.0–3.0 and "Max Zoom" starts at 1.5–4.0, so a user can drag Min Zoom above Max Zoom (e.g. `minZoomAmount=3.0`, `maxZoomAmount=1.5`). In `compute_zoom_amount`, the formula is:
    
    ```
    config.max_zoom_amount + t * (config.min_zoom_amount - config.max_zoom_amount)
    ```
    
    When `min > max`, `(min - max)` is positive, so `t` increasing with spread _increases_ zoom instead of decreasing it. This silently inverts the intended behavior: tight clusters would receive low zoom (1.5×) while spread activity receives high zoom (3.0×) — the opposite of the feature description. There is no guard in the backend either.
    
    Consider adding a UI-level validation that clamps or prevents the inverted state:
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
  2. apps/desktop/src-tauri/src/recording.rs, line 279-280 (link)

    P1 Potential NaN zoom amount when intensity_spatial_scale is zero

    When intensity_spatial_scale == 0.0 and all positions in a cluster happen to share the same normalized coordinate (so max_dist == 0.0), the division 0.0 / 0.0 yields f64::NaN. In Rust, NaN.clamp(0.0, 1.0) propagates NaN, so t becomes NaN, and ultimately ZoomSegment { amount: NaN, … } is emitted. While the slider for this field is not exposed in the UI, intensity_spatial_scale lives in the serialized AutoZoomConfig and can be set to 0 via the settings JSON file.

    Add a guard before the division:

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: apps/desktop/src-tauri/src/recording.rs
    Line: 279-280
    
    Comment:
    **Potential NaN zoom amount when `intensity_spatial_scale` is zero**
    
    When `intensity_spatial_scale == 0.0` and all positions in a cluster happen to share the same normalized coordinate (so `max_dist == 0.0`), the division `0.0 / 0.0` yields `f64::NaN`. In Rust, `NaN.clamp(0.0, 1.0)` propagates `NaN`, so `t` becomes `NaN`, and ultimately `ZoomSegment { amount: NaN, … }` is emitted. While the slider for this field is not exposed in the UI, `intensity_spatial_scale` lives in the serialized `AutoZoomConfig` and can be set to 0 via the settings JSON file.
    
    Add a guard before the division:
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: apps/desktop/src/routes/(window-chrome)/settings/experimental.tsx
Line: 793-810

Comment:
**No constraint enforcing Max Zoom ≥ Min Zoom**

The "Min Zoom" slider has a range of 1.0–3.0 and "Max Zoom" starts at 1.5–4.0, so a user can drag Min Zoom above Max Zoom (e.g. `minZoomAmount=3.0`, `maxZoomAmount=1.5`). In `compute_zoom_amount`, the formula is:

```
config.max_zoom_amount + t * (config.min_zoom_amount - config.max_zoom_amount)
```

When `min > max`, `(min - max)` is positive, so `t` increasing with spread _increases_ zoom instead of decreasing it. This silently inverts the intended behavior: tight clusters would receive low zoom (1.5×) while spread activity receives high zoom (3.0×) — the opposite of the feature description. There is no guard in the backend either.

Consider adding a UI-level validation that clamps or prevents the inverted state:

```suggestion
						<SettingSlider
							label="Min Zoom"
							value={settings.autoZoomConfig?.minZoomAmount ?? 1.2}
							onChange={(v) =>
								handleConfigChange("minZoomAmount", Math.min(v, settings.autoZoomConfig?.maxZoomAmount ?? 2.5))
							}
							min={1.0}
							max={3.0}
							step={0.1}
							format={(v) => `${v.toFixed(1)}x`}
						/>
						<SettingSlider
							label="Max Zoom"
							value={settings.autoZoomConfig?.maxZoomAmount ?? 2.5}
							onChange={(v) =>
								handleConfigChange("maxZoomAmount", Math.max(v, settings.autoZoomConfig?.minZoomAmount ?? 1.2))
							}
							min={1.5}
							max={4.0}
							step={0.1}
							format={(v) => `${v.toFixed(1)}x`}
						/>
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: apps/desktop/src-tauri/src/recording.rs
Line: 279-280

Comment:
**Potential NaN zoom amount when `intensity_spatial_scale` is zero**

When `intensity_spatial_scale == 0.0` and all positions in a cluster happen to share the same normalized coordinate (so `max_dist == 0.0`), the division `0.0 / 0.0` yields `f64::NaN`. In Rust, `NaN.clamp(0.0, 1.0)` propagates `NaN`, so `t` becomes `NaN`, and ultimately `ZoomSegment { amount: NaN, … }` is emitted. While the slider for this field is not exposed in the UI, `intensity_spatial_scale` lives in the serialized `AutoZoomConfig` and can be set to 0 via the settings JSON file.

Add a guard before the division:

```suggestion
        let t = if config.intensity_spatial_scale > 0.0 {
            (max_dist / config.intensity_spatial_scale).clamp(0.0, 1.0)
        } else {
            0.0 // treat scale=0 as "always tight cluster" → max zoom
        };
```

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: "feat(recording): ada..."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant